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1 Introduction 



The Cramer- Rao lower bound on the covariance matrix of an estimator is a clas- 
sical result in statistics, see Cramer 5 and Rao This bound is given by the 
inverse of Fisher's information matrix. For regular statistical models, it is also 
known that the maximum likelihood estimator is asymptotically normal with 
this inverse as the asymptotic covariance matrix. Therefore it is natural to ask 
for conditions of an underlying statistical model that guarantee non-singularity 
of this matrix. In the present paper we are concerned with the Fisher infor- 
mation matrix for (stationary) autoregressive moving average (ARMA) models. 
The information matrix is singular in the presence of common roots of the AR 
and the MA polynomial and vice versa. This fact is considered to be well- 
known in time series analysis, see [12] or |13 for an early discussion of this 
phenomenon, and (Sj for the extension to ARMA models with an exogenous 
input (the ARMAX case). 

In |5] properties of the Fisher information matrix for an ARMA process 
have been derived using contour integration in the complex plane and state 
space realizations of the ARMA process itself. In the present paper we study 
Fisher's information matrix by means of state space realizations for the score 
process and by linking Fisher's information matrix to the Sylvester resultant 
matrix and the Bezout matrix associated with the autoregressive and moving 
average polynomials. 

The role of the resultant matrix has been discussed in various studies in the 
fields of time series and systems theory. For instance, in [J this matrix shows 
up in a convergence analysis of maximum likelihood estimators of the ARMA 
parameters (more precisely in the study of the convergence of the criterion 
function), in Barnett [2] a relationship between Sylvester's resultant matrix and 
the companion matrix of a polynomial is given. Kalman [71 has investigated 
the concept of observability and controllability in terms of Sylvester's resultant 
matrix. Similar results can be found in Barnett which contains further 
discussions and references on these topics. But, it seems that the use of the 
Bezout matrix has not been recognized yet. For ARMA models we will show 
that Fisher's information matrix can be factorized, where one of the factors is 
expressed in terms of the Bezout matrix. Also from this it follows that Fisher's 
information matrix is singular if and only if the AR and MA polynomials have a 
non-trivial greatest common divisor. Singularity of the information matrix can 
thus be interpreted as the result of ovcrparamctrization of the chosen ARMA 
model and of using a model of too high order. In Soderstrom & Stoica |16L pages 
162 ff.] a discussion on overparametrization in terms of the transfer function 
of a system can be found. In a static context. Fisher's information matrix has 
already been studied in jl5j for problems of local and global identifiability. 

The paper is organized as follows. In section |21 the main results are state 
space realizations for the derivatives of the noise process and properties of these 
realizations are presented. In section El we study some properties of the Bezout 
matrix as well as its kernel. Section 0] is devoted to further properties of the 
Bezout matrix, to be exploited in sections O and In the first of these sections. 
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we study singularity of solutions to certain Stein equations with coefficients 
related to the AR and MA polynomials, whereas in section|51all previous results 
are assembled to characterize non-singularity of Fisher's information matrix. 



2 Computations in state space 

Consider the following two scalar monic polynomials in the variable z. 



a(z) ~ + aiz^ 
c{z) = z^ + ciz'^^^H \-Cq. 

By a and c we denote the reciprocal polynomials, so a{z) = z^a{z^^) and 
a{z) = z'^c{z~^), and also write = (ai, . . . , Op) and = (ci, . . . , Cq). Usually 
no confusion between the notation a for the polynomial and vector will arise, 
but sometimes we will write a(-) when a polynomial is considered. 

Consider the stationary ARMA(p,g) process y that satisfies 

a{L)y^c{L)e (2.1) 

with L the lag operator and e a white noise sequence. We make the assumption 
(to give the expressions that we use below the correct meaning) that y is causal 
and invertible, i.e. both a and c have only zeros outside the unit circle (equiva- 
lently, a and c have only zeros inside the unit circle). 

Let 9 = (fli, . . . , Op, ci, . . . , Cg) and denote by s^' the derivative of with respect 
to di. Then we obtain by differentiation of H2.1|) the formal expressions 

s? ^ (2.3) 

Let £t = St{9) denote the row vector with elements ef* and = ej . See 
section for the relation with the stationary Fisher information matrix of the 
ARM A process y. We introduce some auxiliary notation. Write for each positive 
integer k 

Uk{z) = (l,z,...,z'=-i)^ 

uUz) = (z'=-i,...,l)T = zfe-iy,(z-i). 

Let us compute the transfer function t{z) that relates ^ to e by replacing L with 
z~^ in equations H2.2|l and 12.3|1 . Here z^^ represents the forward shift. One 
obtains from and 

In [5j we have investigated certain controllable or observable realizations of the 
ARM A process y. There we have also briefly outlined a procedure without 
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detailed proofs to obtain from these realizations also realizations for the process 
e. We repeat the conclusions, but give in the present paper a short proof of them, 
based on transfer function considerations, without using the realizations of the 
ARMA process y itself. Let e„ be the first basis vector of the Euclidean space 
K". When no confusion can arise (often in the proofs), we often simple write 
e, which we will also use as the notation for the first basis vector in Euclidean 
spaces of different dimensions. By J we denote the forward shift matrix, Jij = 1 
if i = J ' + 1 and zero else. 

Similarly, we denote by / the identity matrix of the appropriate size and 
stands for the zero vector or matrix of appropriate dimensions. Occasionally 
these matrices and vectors will have a subscript, when it is necessary to indicate 
the sizes. 



Letg = a(z)c(z) = zP+^ + Er=i3' 



-,p+q-i 



and g the vector g — {g. 



P+9' 



,9i) 



Likewise we write g{z) — a{z)c{z) and g — {gi, . . . , gp+q). 



The Sylvester resultant matrix i? of c and —a is defined as the (j) + q) x {p + q) 
matrix 



i?(c, —a) 



Rp{c) 
-Rq{a) 



where Rp{c) is the q x {p + q) matrix 



Epic) = 



Vo 



Cl 

1 



Cl 





Cq 
Cl 



and Rq (a) is the q x {p + q) matrix given by 



Rq{a) 



/I 



Vo 



ai 
1 



ai 









Op 

ai 



In the presence of common roots of a and c the matrix i?(c, —a) becomes sin- 
gular. Moreover it is known (see e.g. |17l page 106]) that 



det i?(c. 



i=ij=i 



OLi) 



(2.5) 



where the Ui and the 7j are the roots of a and c respectively. 
Next we introduce the matrices F and G defined by 



F = 



J — epfl 








(2.6) 
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and 

G = J-ep+,g^. (2.7) 



Lemma 2.1 Let F and G he as in f2. b]) and \2/J^ . Then the following relation 
holds. 

R{c,~a)G ^ FR{c,~a). (2.8) 

Proof. The easiest way to see this, is to muhiply both sides of this equation 
on the right with Up+q{z). Then we compute on the left hand side the product 

R{c, -a){J - ep+gg^)up+g{z) = R{c, -a){up+q(z) - g{z)e) 

c{z)up{z) - g{z)ep 
-a{z)uq{z) + g{z)eq 

The computations on the right hand side are of a similar nature and an easy 
comparison yields the result. □ 

We now present the first realization of the process i. 

Proposition 2.2 The process ^ — can be realized by the following stable and 
controllable system 

Zt+i = GZt + eet (2.9) 
6 = CZt, (2.10) 

where G is as in \2. 7| ) and G = R{c,—a). This system is observable iff the 
polynomials a and c have no common zeros. 

Proof. Let us compute the transfer function r of the above system. Standard 
computations show that (z — G)^^e = |p^M*^^(z). The trivial observations 
Rp{c)up^q{z) = c{z)up{z) and i?g(a)u*_|_^(z) = a{z)u*{z) then lead to the con- 
clusion that t{z) — C{z — G)~^e is exactly the same as in (|2.4|) . The system 
is obviously controllable. The observability matrix of the system involves prod- 
ucts of the form i?(c, —a)G^ {k = 0, . . . ,p + q — 1). In view of lemma |0] these 
can be written as F^R{c, —a), from which the assertion follows. Stability is an 
immediate consequence of the assumptions on the polynomials a and c. Indeed, 
the characteristic polynomial of G is g = ac, which has its zeros inside the unit 
circle. □ 

An alternative (observable) realization of the process e is given in the next 
proposition. 

Proposition 2.3 The process £^ = is the state process of the stable system 
given by 

^t+i^F^t + Bsu (2.11) 

where F is as in \2.(^) and B = { J . This system is controllable iff a and c 
have no common zeros. 
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Proof. Again the proof that this reahzation produces e boils down to com- 
puting the transfer function, like we did in the proof of proposition 12.21 The 
computations needed for this have been encountered there, so we skip them. To 
explain the statement on controllability, we consider the equation (w, v are row 
vectors and A is an arbitrary complex number) 



where Fa = J ^ Cpa and ~ J — CqC ■ This equation is equivalent to 
u{Fa — A) = 0, v{Fc — A) = and {u — v)e = 0. We first consider the case 
where a and c have no common zeros. Suppose that m = 0. Then we have that 
v{Fc — A) = and ve = 0. Since (Fc, e) is controllable, v must be zero as well. 
Therefore we will assume that there is a nonzero solution u. Then A must be 
a root of d{z) = 0. If u = 0, then we also have ue — 0. This situation cannot 
happen since {Fa, e) is a controllable pair. So we have to assume that v is not 
zero, but then A is also a root of c{z) = 0. It then follows from the above that 
this cannot happen. Hence [F, B) is controllable. In the other case a and c have 
a common zero A. In this case u is the row vector (1, ai(A), . . . , ap_i(A)), where 
the a.i are the Horner polynomials, defined by ao{z) — 1, afe(z) — zdfe_i(z) + 
and we have a similar expression for v. One obviously then also has {u — v)e — 
and hence the system is not controllable. Stability follows upon noting that the 
characteristic polynomial of F is equal to g. □ 

Remark 2.4 By lemma ITTl the realization of proposition 12 . 31 is connected to 
the one in proposition 12 . 21 in a very simple way. Starting from equation (|2.1U|) . 
one obtains 

^t+i = R{c, -a)Zt+i 



Remark 2.5 Notice that the realizations of propositions 12 . 31 and 12 . 21 illustrate 
the well known fact that it depends on Eg for s < t only, and hence is stochas- 
tically independent of £t- 

3 The Bezoutian 

We follow the notation of Lancaster & Tismenetsky ,11, . Recall the following 
definitions. In this section and henceforth we assume that p and q are taken to 
have a common value, denoted by n, to yield many of the subsequent expression 
meaningful. We consider polynomials a and h given by a{z) = X]fc=o ^kZ^ 
^{z) = 'Y^^=o^kZ^ ■ We will always assume that the constant term — 1 and 
likewise for h and other polynomials. 

Consider the Bezout matrix B{a, b) of the polynomials a and b. It is defined by 




R{c,~a){GZt + eet) 
F^t + Bet. 
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the relation 

a{z)b{w) — a{'w)b{z) = {z — 'w)un{z)^ B[a,b)un{w). 

We also often call this matrix the Bezoutian. Introduce for a given complex 
number 6 the matrices Ud, as follows. 



Us 



( 1 







Vo 



■• 

.0-0 1/ 

We also need the inverses of the matrices U^. These take the form 

/I 0\ 

1 

a2 J, ■• 



Ts 





1/ 



Observe that matrices C/^ and commute, as well as and T^. 

Consider again a and 6, n-th order polynomials with constant term equal to 1. 

Let (1 — aiz) be a factor of a(z) and (1 — li\z) be a factor of h{z). Of course, 
a\ and /3i are zeros of a and c respectively. Write a{z) = (1 — a\z)a-\{z) and 
h{z) = (\— (5xz)h-\{z). Continuing this way, for ai, . . . , q;„ we define recursively 
a_(fe-i)(z) = (1 — afez)a_fe(-z) and polynomials 6_fc similarly. We also put 
a^iz) = a{z) and bo{z) = b{z). The following proposition is not completely 
necessary for what follows, but may be of independent interest. 

Proposition 3.1 With the above introduced notation we have 



a{z)b{w) — a{w)b{z) 



w 



={1 - aiz){l - piw 



a-i{z)b-i{w) - a-i{w)b-i(z) 



w 



+ (/?! - ai)a-i{w)b-i{z). 



(3.1) 



In terms of the Bezoutian this is equivalent to the (non-symmetric) decomposi- 
tion 



B{a, b) = Ua 



B(a_i,6_i) 







(3.2) 



with such that a^^Un{z) = a-i{z) and b^^ likewise. 
Iteration of this procedure gives 



a{z)b{w) — a{w)b{z) 



z — w 



i{z)b{w) ^(/3fe - ttfc) 



fe=i 



a-kiw)b-kiz) 

0_(fe_i)(2:)6_(fc_i)(u;) 



, (3.3) 
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which gives the following expansion for the Bezout matrix 



Bia, b) - ^(/3fc-afc)C/„, • • • U^,_,Up,^, ■ ■ ■ f/0„eeTf/T . . . Ul_^UZ^^^ ■ ■ ■ C/J„. 
fc=i 

Proof. Equation follows from elementary computations. To prove ()3.2|l 
we premultiply both sides of the equation by w„(z)^ and postmultiply them by 
Un{w). The obtained left hand side then is obviously equal to the left hand side 
of H3.ll) . To show that the right hand sides coincide one uses that Un{z)^ Ua^ = 
(1 — aiz){un-i[z)^ ,Q) + (0, . . . , 0, Then the assertion easily follows from 

the definition of i?(a_i,5_i). To prove the other assertions, we proceed as 
follows. First we show how the right hand sides of equations 1)3. 3|l and (|3.1II are 
related. We pre-multiply the right hand side of H3.1|l by u„(z)^. The important 
key relation is 

k~l n 

UnizyUa,---Ua,_^Up,^^---Uf3„e=Y[{l-ajz) Yl 

j=l J=k+1 

which is easily shown to be true. Of course the right hand side of this equation 
is nothing else but 

-b-k{z). 



-(fc-i) 



(^) 



Then post-multiplication of the obtained expression by u„(u)) obviously results 
in the right hand side of (|3.3(l . 

We now show by induction that this is equal to w„(z)^i?(a, b)un{w). Let A{z) = 
(1 — a{)z)a{z), B{z) = (1 — /3Qz)b{z). Define 

A-k{z) = — — 

and define B_k{z) likewise [k = 0, ...,n). We also let Ai{z) = A{z) and 
Bi{z) = B{z). We will use the following trivial identities. For fc = 1, . . . , n we 
have A_k{z) = a„fe(z) and = b^k{z)- 



A{z)B{w)Y,{Pk - ak 

k=0 



A-(k-i){z)B^(k-i){w) 



(/3o - ao)a{w)b{z) + 



(1 _ aoz)il - Ma{z)b{w) j^iPk - a.) ^z^H^^ , 

^ a~{k~i){z)b-(k-i)\w) 



k=l 



A{z)B(w)-A{w)B{z) 



In view of H3.1|l and the induction assumption, this equals 
The proposition has been proved. □ 



z—w 
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Corollary 3.2 Let cj) be a common zero of a andb. Then a{z) — {\~(j)z)a^i{z) 
and b{z) = (1 — and 

i?K^) = t/0(''^"-J''-^^ o)^;. (3.4) 

Proof. This is a straightforward consequence of the previous proposition. □ 

ProDOsition l3.1l can be used to show the well known fact (see ^2 theorem 13.1] 
or [HJ theorem 8.4.3]) that the Bezout matrix B{a,h) is non-singular iff a and b 
have no common factors. We use corollarv l3.2l to give a description of the kernel 
of the Bezout matrix. 

Corollary 3.3 Let 71, ... , 7^ be all the common zeros of a and b, with multi- 
plicities ni^ ... ^Um- Let I be the last basis vector of andputv^. = {T^^J^~^)'^i 
for k = 1, . . . , m and j — 1, . . . , rife. Then ker _B(a, 6) is the linear span of the 
vectors w^. 

Proof. First we have to show that the vectors v^, are independent. Explicit 
computation of these vectors show that, after multiplication with P, they are 
columns of the confluent Vandermonde matrix associated with all zeros of a, 
from which independence then follows. For j = 1, it follows immediately from 
corollary 13.21 that the belong to the kernel of the Bezout matrix. When (j)k 
is common zero with multiplicity j > 1 we can factor the matrix i?(a_i,6_i) 
in H3.4|l like B{a,b), but with one dimension less. However, one can then show 
that also 

where for instance the O-matrix in the lower right corner now has size 2x2. 
Continuation of this procedure leads to 

for J = 1, . . . , Uk- Since the last j columns of B{a, b) are thus zero vectors, one 
immediately sees that B{a, 5)u^ = 0. The proof is complete upon noticing that 
the kernel of the Bezout matrix has dimension equal to ni + • • • + Um (cf 
theorem 8.4.3]). □ 

Remark 3.4 For more applications of confluent Vandermonde matrices to the 
analysis of stationary ARMA processes, we refer to |10| . 

4 The Bezoutian and the ARMA polynomials 

In this section we continue to study some properties related to the Bezout 
matrix, which (aimed at applications in section we express in terms of the 
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ARMA polynomials a and c that define the process y of equation (|2.1|l . For a 
polynomial a(z) = X]fe=o '^fe-^'^ the matrix S'(a) is given by 



/ ai 

02 



a2 



\a„ 
and S'(d) is given by 







an-2 



S{a) 







ao 



0/ 



As before we will work with polynomials a whose constant term oq = 1. Notice 
that S{a) is connected to the reciprocal polynomial a, a{z) = "Y^^^o o,n-kZ^ , 
as is S{a) to a. Let P be the 'anti-diagonal identity' matrix in R"^", so with 
elements Pij = 5i^n+i-j- On a TbepZite matrix M pre- and postmultiplication by 
P results in the same as transposition: PMP = . We will use this property 
mainly for the choice M = J, the shift matrix. 

We continue under the assumption that the polynomials a and c have common 
degree n. One of the possible relations between the Sylvester matrix R{c, —a) 
and the Bezoutian B{c, a) is given below. 

Proposition 4.1 The matrices R{c, —a) and B{c,a) satisfy 



P 
PS{a)P PS{c)P 



R{c, —a) = 









B{c, a) 



PS{c)P S{c) 
/ 



Proof. This relation is just a variant on equation (21) on page 460 of 11 and 
can be proven similarly. □ 



( P 
\PS{a)P PS{c)P 



We will use the short hand notation 
M (c, a 

and 



(PS{c)P S{c] 
\ / 



(4.1) 



(4.2) 



Notice that both M(c, a) and N{c) are nonsingular if ao ^ and co (which 
is our case, since we always work with ao = co = 1). 
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Theorem 4.2 Let F, G, M{c, a) and N{c) be as in equations fE7| ), fTTTp 

and j^.iil ). T/ie following identities hold true. 



N 1 fP(J-ea^)P 



G 



1/ 



(4.3) 



and 



N{c)FN{c)-^ = 



fPiJ-ea'^)P 



\ ee 
Moreover we have the relation 

\ fl 



J-ec^ 



=: F, 



G 



M 



B[c,a) 



B[c,a) 



F, 



N- 



(4.4) 



(4.5) 



Before giving the proof of this theorem we formulate a few technical lemmas 
that will be of use in this proof. 



Lemma 4.3 The following two equalities hold true. 



PS{c)P{J - ec^) = {PJP - ce^)PS{c)P = 



1 

Cl 



\ 



C„-2 
~Cr 





1 Cl ■ 

Vo ••• • 

Proof. Compare to the analogous statement in JT], page 455. □ 

Lemma 4.4 Let g{z) = a{z)c{z) = Y^kLo9kZ^ '^"■^ 9 = {9^t9^)j ^^^^ 9^ = 
{gi, . . . , gn) and g^ = (gn+i, • • ■ , 92n)- Then the following identities hold true. 

S{c)Pe = e. 



a^S{c)P = (ffi - c) 



S{c)Pa = g'^. (4.6) 
Proof. This is a straightforward verification. □ 

Along with the matrices S{c) and S{c) we also use the Hankel matrix S{c) £ 
]^(n+i)x(n+i) defined by 

/C„ Cl 1\ 

: 1 



Cl 1 

VI 



0/ 
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Lemma 4.5 One has 



JS{c) + ec^P = (/ 0) S{c) Q 



In particular the matrix JS{c) + ec^ P is symmetric. 
Proof. The following relations are immediate. 

5(c) = (0 l)S{c)(^,) (4.7) 



and 



(4.8) 



ec^P=(e 0)5(c)Q. 
Use equations (14. 7f) and (|4.8|) to write 

JS{c) + ec^P = J (0 /) S{c) (^^^ + (e O) S{c) 

= ((0 j) + (e 0)))5(c)Q, 

from which the result follows. □ 

Proof of theorem 14. 2L Compute the two products M{c, a)G and GmM{c, a) 
to get respectively 



/ P(J-ea^) 
[pS{a)P{J - ea^) PS{c)P{J - ec^; 



(4.9) 



and 

(4.10) 



P(J-ea^) 
(c - a)e^ + {PJP - ceJ)PS{a)P {PJP ~ ce^)PS{c)P 

Clearly we only have to look at the 21- and 22-blocks. Comparing the 22-blocks 
is just the content of lemma 

We focus on the 21-blocks. Use lemma ITTSl again to write PS{a)P{J — ea^) (the 
21-block of iHSJ) as 

{PJP - ae^)PS{a)P = PJS{a) - ae^ P = PJS{a) - ce'^ P + (c - a)e^P = 

PJS{a) - ce^PS{a)P + (c - a)p7P, 
which is just the 21-block of H4.10|l . This proves the identity 1)4.^(1 . 

Next we prove (14.411 . Write F — {'^ ) . Work out the products 

\ ee P J J 

N{c)F and FnN(c) to get respectively 

^PS{c)P{J - eg^^) + 5(c)eeTp ^PS{c)Peg^'" + S{c)J^ ^^^^^ 
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and 

fP{J^ea^)S{c)P P{J-ea^)PSic)\ 

y ee^PS{c)P ee^Sic) + J-ec^J- ^ ' 

Compare now the corresponding blocks in these two matrices. We start with 
the 11-block of (jHH)- Write it as 

P{S{c)PJP + Pce^)P - Peg^^ = P{S{c)J^ + Pce^)P - Peg^^ 

and use the symmetry asserted in lemma to get 

P{JS{c)+ec^ P)P-Peg^^ = PJS{c)P+e{c^ ~g^^ ) = PJS{c)P-Pea^ S{c)P, 

which equals the 11-block of H4.12|l . 

Next we consider the 12-blocks. Start with H4.11|l : 

~PS{c)Peg^^ + 5(c) J = -Peg^^ + S{c)J = -Pea^PS{c) + S{c)J, 

where the last equality just follows from H4.6|) . Since 5(c) J is symmetric it is 
equal to J^5(c) = PJPS{c). Hence 

-Pea^PS{c) + 5(c) J = P{J - eaJ)PS{c), 

which is equal to the 12-block of 14.12|l . Comparison of the other blocks is triv- 
ial. 

Finally we prove H4.5|l . Remember that GR{c,~a) = R{c,—a)F (proposi- 
tion [THl. Write B for ( , ) . Then we have the string of equalities 
\0 B{c,a)J 

GmB = GmM{c, a)R{c, -a)N-^ = M(c, a)GR{c, -a)N;:^ = 

M{c, a)R{c, -a)FN-^ ^ BN^FN-^ ^ BFn- 
This proves the last assertion of the theorem. □ 

5 Stein equations 

We start this section with considering two Stein equations that involve the 
matrices Fm and Gn of equations (|4.3|) and (|4.4|) . 

Proposition 5.1 Let ej, ~ [e^P,0]^ and let H{9) and Q{9) be the unique 
solutions to the following Stein equations 

H = GMHGli + epel (5.1) 
Q = FNQF^ + epel. (5.2) 

Then Q{9) is strictly positive definite. Moreover, H{6) and Q{9) are related by 

i?(c%)) «(^) 5(c%))- (^•^) 



12 



Proof. To show that Q{0) is strictly positive definite, it is sufficient to show 
that the pair {FN,ep) is controllable. Let T = For computational 



.0 /y 

reasons it is more convenient to show controllability of the pair {A,b), where 
A = TFmT~^ and b = TF^. Observe that b is the first standard basis vector in 

/ J _ gfjT p \ 

R^", whereas ^ = I to t t I ■ If oi^^ computes the controllability 

matrix {b, Ab, . . . , then standard calculations lead to the fact that this 

matrix is upper triangular with only ones on the diagonal. Hence it has full 
rank. By theorem 8d.66 of 0] the matrix Q{6) is strictly positive definite. 
Multiply equation l|5.2(l with Q = Q{9) on the right and on the left by the 

symmetric matrix T = ^ and put H = TQ{9)T. In view of rela- 

tion (|4.5(l we then obtain equation (|5.1|l . Hence H must be equal to H{6) by 
uniqueness of the solution. This shows H5.3|l . □ 

Corollary 5.2 The matrix H{9) is non-singular iff the polynomials a and c 
have no common factors. 

Proof. The matrix Q{0) is non-singular and B(c,a) is singular iff the polyno- 
mials a and c have no common factors. □ 



Remark 5.3 If the polynomials a and c have a common factor (1 — (j)z), then 
the expression for B{c, a) of equation (|3.4fl can be applied to obtain a rank 
factorization of H{9). 

Along with the matrices H{0) and Q{9) of proposition 15.11 we will also work 
with the matrices I{6) and P{6), defined in 

I{e) = M{c, ay^H{e)M{c, a)-^ (5.4) 
P{e) = N{c)-^Qi9)N{c)-\ (5.5) 

where M(c, a) and N{c) are as in (|4.HI and (|4.2|l . In view of equations (|4.3|l 
and H4.4|l and proposition we have that I (9) and P{9) are solutions to the 
Stein equations 

/ = FIF^ + BB^ (5.6) 
P = GPG^+ee'^. (5.7) 



Corollary 5.4 The matrix P{9) is non-singular and the matrix I{9) is non- 
singular iff the polynomials a and c have no common factors. 

Proof. Since the matrices M{c, a) and N{c) are non-singular, the results follows 
from proposition 15 . II and corollarv l5.2l □ 
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6 Fisher's information matrix 



We consider again the ARMA process y defined by (|2.1() . Let the variance of the 
white noise sequence be . Assume that the process is stationary. TraditionaUy, 
the Fisher information matrix is defined as the covariance matrix of the score 
function. Let us assume that the process is also Gaussian and that cr^ is a 
known constant. If we have observations yi, . . . ,yn, the log hkehhood £n{9) is 
then essentially given by 

£„(^) = ^loga^-^^e.(^f. 

t=i 

The score function, by definition the gradient of ^„(0), is then given by 
1 " 

Lie) ^ --Y.etie)e,{9). (6.1) 
t=i 

Remember that for et{0) we have the realizations ()2.9|l and (|2.11|) . In remark [T5l 
we observed that et{0) is (stochastically) independent of £t, and we also have 
£^((6*) — (expectation taken w.r.t. the distribution under 9). The covari- 
ance matrix /„(0, cr^), by definition the covariance matrix of £ni9), can then be 
computed as 

Ud,a^) = Cov(i„((?)) = EL{d)'^U9). 

It then follows from (|6.1|1 and the above mentioned independence that 

^ n ^ n 

In{9,a^) ^ — y^Eet{9fEet{9yet{9) = — V Eet(0)^et(0). 
t=l t=l 

For it{9, cr^) :— E et{9)^ it{9) we get from equation H2.11|l and the independence 
of et(0) and et{9) the recursion 

it+ii9,a^) ^ Fiti9,a^)F'^ + a^BB^ . (6.2) 

Under the stationarity assumption we have it+i{9, cr^) = it{9, cr^) and we simply 
write i{9,a'^). Hence /„(0, cr^) — nl{9,a'^), where 

I(9,(j^) ^ ^Ei(9,a^). (6.3) 

Without stationary initial conditions, but still with e a Gaussian white noise 
process, one can show (but this is not relevant for the present paper) that 
-In{9,(J^) — > I{9,a'^). Hence the matrix I{9,(t^) is also relevant in a non- 
stationary situation. We call I [9, cr^) the asymptotic Fisher information matrix. 
Summing up intermediate results, we obtain the following theorem. 
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Theorem 6.1 The asymptotic Fisher information matrix 1(9, cr^) of the ARMA 
process defined by f2.1\) is the same as the matrix 1(9) defined in equation ^5.4]) - 
Hence it is the unique solution to the Stein equation 

I ^ FIF^ + BB^ , (6.4) 

and thus independent of . Moreover this matrix is non-singular iff the poly- 
nomials a and c have no common factors. 

Proof. From equations Ht).2|l and and the stationarity assumption, one im- 
mediately sees that 1(9, a'^) satisfies (|6.4f) . which is just equation H5.6|) . Hence 
the matrices I{9,a'^) and I {9) are the same and the characterization ol non- 
singularity is nothing else but corollarv l5.4l □ 

The conclusion of this theorem has been proved in 9 by different means, in- 
volving representations of the Fisher information matrix as an integral in the 
complex plane and the following lemma of which we give an alternative proof. 

Lemma 6.2 Let 1(9) be the Fisher information matrix and P{9) as in 1^5. 5\) . 
Then the following factorization holds. 

I{9) ^ R{c,-a)P{9)R{c,~ay (6.5) 

Proof. This follows from proposition 14.11 combined with equations (|5.3() . (|5.4|l 
and (ESI. □ 

Since the matrix P{9) is non-singular, also lemma |^| illustrates the fact that 
I{9) is non-singular iff a and c have no common factors. Moreover, looking 
at equation (|5.6(l . we see that / = I {9) is non-singular iff the pair {F,B) is 
controllable. But the controllability matrices TZ{F, B) and Tl{G, e) satisfy the 
easily verified relation TZ{F,B) — R{c,~a)TZ{G,e). Since the matrix TZ{G,e) 
has full rank, we see that [F, e) is controllable iff R{c, —a) is non-singular, which 
leads to another way of showing the conclusion of theorem l6.1l 
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